Multi-task learning strategies for a recurrent neural net in a hybrid tied-posteriors acoustic model

An important goal of an automatic classifier is to learn the best possible generalization from given training material. One possible improvement over a standard learning algorithm is to train several related tasks in parallel. We apply the multi-task learning scheme to a recurrent neural network estimating phoneme posterior probabilities and HMM state posterior probabilities, respectively. A comparison of networks with different additional tasks within a hybrid NN/HMM acoustic model is presented. The evaluation has been performed using the WSJ0 speaker independent test set with a closed vocabulary of 5000 words and shows a significant improvement compared to a standard hybrid acoustic model if gender classification is used as additional task.